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THE CRACKED CAKE OF EDUCATIONAL CUSTOM 
AND EMERGING ISSUES IN EVALUATION 



Dan Lortie 

The changes taking place in American public schools today 
have a familiar ring. Contact replaces isolation as new social 
groups (business, federal agencies) engage themselves in school 
affairs. Heterogeneity displaces homogeneity when school staffs 
expand to include people from different occupations (social work, 
psychology, library work) and teachers are more diverse in social 
and educational backgrounds. A hierarchical, paternalistic 
authority system is challenged by subordinates who clamor for a 
say in decision-making. Interaction within the subsystem quickens 
as new kinds of buildings and work patterns (team teaching for 
example) eliminate the walls separating fellow teachers and fellow 
students. Age-old pedagogical conventions are discarded as stu- 
dents are turned loose to teach themselves with the help of com- 
plicated, expensive machines. All these juxtapositions have been 
witnessed before by anthropologists and sociologists studying cul- 
tural and social change; the analogy to processes of modernizing 

societies is striking. In Walter Bagehot's phrase, ’’the cake of 

2 

custom is cracking.” 

Cultural and social change involves shifts in how people 
assign value to various parts of their world; ambiguities arise 
as old certainties melt. This paper explores this process as 
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educational evaluation confronts a system in transition; it focuses 
on issues which arise as a consequence of change.^ Such analysis 
requires more than mere assertion that changes are taking place- - 
the trick is to trace their specific effects . There is the temp- 
tation, often yielded to by sensational journalists, to see a revo- 
lution in every protest rally. Conservatism seems indicated. I 
shall, therefore, limit my observations to trends which are already 
visible and, in sketching out probable effects, eschew long lines 
of inference. It is, in fact, very doubtful whether social science 
theory permits us to gauge anything more than such first-order 
effects. Yet it is interesting that the implications of current 
educational change are such that even a prudent approach produces 
a set of rather complicated possibilities. 

Organizational Trends and Evaluation 

Using the local school district as our point of reference, 
we can classify organizational trends occurring today as "external” 
and '’internal.” We begin with events taking place in the external 
system. There seems little doubt that the augmented roles of the 
federal government, business corporations, and universities will 
have important effects on the public schools. 

The activities of the federal government erode educational 
tradition. We can see this as the government legitimates and 
diffuses that set of ideas symbolized by the phrase "research and 
development.” This conception of educational practice stresses a 
core idea of rationality in governance - -it applies scientific ways 
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of thinking to the appraisal of alternatives and to the making of 
decisions. This viewpoint is not new to university professors. 

Yet as a statement of official government policy, backed by public 
tax monies , it is novel for the public schools . The research and 
development viewpoint has migrated from universities and industry 
to school boards justifying their claims for federal grants. It 
becomes part of the working reality of school officials because it 
is built into the rhetoric of applying for funds and undergrids 
the logic of allocation used by federal agencies. We need not 
argue that school officials understand it fully to argue that it 
influences their actions. Thoughtways can affect organizational 
behavior even where understanding is incomplete . 4 It is becoming 
routine for federal agencies supporting new programs to require 
recipients to build in evaluation procedures . Such requirements 
force compliance to the new ideology and undermine belief in 
tradition as the warrant for educational practice. 

Federal activities in education are conducted by a variety of 
agencies featuring a variety of primary objectives. It seems that 
once an agency takes on the foundation function, its officials 
begin to act like foundation men; they prefer to back undertakings 
which are original and, if possible, dramatic. Thus, new funds, 
coupled with diverse sponsors who bend toward the novel, produce 
an increased number of institutional options and thereby add to 
the alternatives confronting decision-makers in local school 
districts. Such an increase in options reinforces i. otions of 
rationality conveyed by the research and development ideology. 

The necessity to make choices forces people to attend to the 
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grounds for choice. The larger the number of alternatives, further- 
more, the more those who would prefer to "stand pat" must defend 
their inactivity as a choice in and of itself. This is not to say 
that the availability of options carries assurance that all or even 
most will be adopted by local school officials. But when we re- 
call the potency of incentives possessed by the federal government 
(i.e., large amounts of new monies, a capacity to give national 
publicity to selected school systems), it seems reasonable to ex- 
pect at least some of the new approaches to enter school decision- 
making. It is theoretically possible, moreover, that some highly 
innovative school systems will make radically divergent initial 
choices (e.g., Pittsburgh's commitment to educational parks) and, 
constrained in the selection of subsequent solutions, will ulti- 
mately branch into highly divergent overall solutions. It is 
probably to early for us to conduct empirical studies of radical 
branching, but it is a possibility well worth keeping in mind. 

Business corporations dealing with educational matters will 
add to the number and range of instructional practices and solu- 
tions j individual firms, in fact, wil] lo so or go out of business. 
The new firms combining publishing and electronic resources, more- 
over, frequently have vast financial resources available for devel- 
opment, production, and sales. Competition between firms is likely 
to be firece, particularly as each battles to get and hold as large 
a share of the market as possible. Recall as well that when busi- 
nessmen talk about "merchandising," they are describing effective 
techniques for disseminating practices. If we can assume that no 
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single firm or coalition of firms monopolizes sales, the outcome 
will be further differentiation among school systems as patterns 
of purchasing and implementation vary. 

It is many years since Veblen showed that universities act 
like business corporations in struggling for prestige, wealth, and 
influence. They, like business organizations, are under pressure 
to come up with differentiated ’’products” for the educational mar- 
ket. Universities have their own resources in countering compe- 
tition from government and business; they export highly trained 
persons as well as ideas. Graduates of a particular university 
can be uniquely competent in implementing and refining an approach 
developed at their institution; professors acting as consultants 
widen the institution's sphere of influence. Competition between 
universities is, of course, softened by cooperation among special- 
ists from different institutions (e.g., the professor-developed 
curricula) , but the net effect is similar for local school offi- 
cials . They confront not only diverse university programs but 
demands for more and more student time issued by competing hands 
of university scholars. Professors engaged in public school 
affairs produce alternatives which must be considered by those 
governing school districts. 

Changes in the external system, then, point to greater pres- 
sures on local decision makers to deal with ever more possible 
lines of action. School officials will probably look for ways to 
reduce those pressures; inquiry would probably reveal that struc- 
tures are being constructed now to filter and contain innovative 
forces^ yet the external system, primarily because it is external 
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and largely outside the power system of local officials, can stay 
with its self-appointed function of generating new ways to keep 
school. School officials, whether they wish change or prefer 
continuity, will have little choice but to examine an expanding 
number of instructional approaches in the years ahead. 

The examination of new alternatives will require considerable 
increases in the amount of evaluative activity carried on by school 
personnel. The justification of a particular choice requires 
comparisons between alternatives and the explication of general 
grounds for choice; effects on students and the rating of effects 
as more or less desirable will be difficult to avoid. Thoughtful 
school board members, administrators, and teachers will be skeptical 
of claims made by sponsors of any given approach. Less thoughtful 
colleagues may find that the public expects them to appear as if 
they are giving careful consideration to new possibilities. One 
does not need to be a specialist in evaluation to realize that 
tradition is no guide in choosing between competing novelties. 

The evaluative load will increase most dramatically where 
school systems undertake large-scale changes. The simultaneous 
introduction of innovations creates a special and demanding eval- 
uative problem, for one must take account of local circumstances 
and interaction effects among innovations. For example, a school 
system might decide to combine educational parks with computer- 
assisted instruction. What affective outcomes flow from imperson- 
alities associated with man-machine systems and sharp increases in 
the number of fellow students? Situations of this sort do not 
permit local officials to apply evaluations developed elsewhere; 
they must do their own digging. 




7 



Two major trends are taking place in the internal system of 
public schools, and both are likely to have serious implications 
for the conduct of evaluation. The first, functional differentia- 
tion, develops quietly and may go unnoticed. The second, ’’teacher 
militancy,” hits the headlines almost daily. Both trends, however, 
seem to share a common effect. They weaken familistic and pater- 
nalistic conceptions of authority relationships among people work- 
ing in schools. 

The history of American school organization is largely the 
story of increasing specialization in the knowledge to be trans- 
mitted and in the tasks of those engaged in transmitting that 
knowledge. Contrast, for example, the one -room school house of 
the nineteenth century with its modern counterpart, the rural 
regional school. Grades, subjects, and teaching tasks have all been 
subdivided. Today, specialists counsel students, supervise their 
health, store and distribute books, purchase and distribute audio- 
visual equipment , and visit families with problems. These special- 
ists, moreover, have separate occupational associations concerned 
with ’’professionalizing” each subfield. One influential spokesman 
has called for the creation of a national system of specialty 
boards for various categories of teachers.^ 

Yet the division-of- labor we see today may prove to be but a 
pale prologue to much greater differentiation in the future. 
Functional differentiation among school professionals, so far, has 
tended to be differentiation among equals. But now we see new 
forms of stratification being introduced as specialists of lower 
or higher status are hired. Some school systems, for example, are 
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employing teacher aides whose credentials make it very unlikely 
that they will ever move into teacher ranks . Team teaching 
arrangements in some places, on the other hand, involve higher 

*1 O 

status for team leaders and senior teachers. New technologies 
bring specialists, sometimes of higher rank, in their wake; we now 
have television teachers, programmers of instruction, and computer 
experts . Similar things are happening in central offices where 
superintendents look for men to specialize in relationships with 
the federal government, experts in collective bargaining, and men 
who can design program budgets. The pace of role differ entiation 
is quickening . 

Problems of communication and conflicts in orientation occur 
more readily when systems become internally differentiated . 11 
Occupational differentiation produces specialisms not only of 
skill but of perspective, of moral outlook. Additional layers 
of authority in organizations complicate communication within by 

13 

producing more blockages in the flow of information and affect. 

Thus, we can look ahead to schools and school systems where people 
of diverse outlook and rank find it harder to agree on instructional 
matters. We expect that mechanisms will be developed to cope with 
this problem. Building such mechanisms, however, requires consid- 
erable time, and it is debatable whether they ever attain the easy 
consensus associated with earlier social homogeneity. 

Perhaps "teacher militancy" is a special case of role differ- 
entiation; in any event, it is clear that teacher demands are pro- 
ducing controversy over instructional matters as well as salaries 
and working conditions. The New York City strike is a recent 



example. There the union bargained hard and long over whether the 
More Effective Schools Program would receive additional financing. 
Nor is New York unique, for state after state is making legal pro- 
vision for the participation of teacher groups in setting school 

policy . ^ Other categories of school workers are also agitating 

15 

for more influence in instructional matters. It is ironic that 
the external system should begin to produce increased options at 
that point when overt conflict on instructional policies emerges 
within the internal system. 

Evaluation will undergo alterations where instructional policy- 
making is colored by conflict. Spirited advocacy by opponents will 
make policy deliberations more like courtroom trials and legislative 
battles. Protagonists, eager to cloak their positions in the garb 
of educational superiority, will buttress their beliefs with eval- 
uations . Overt conflict may lead to debates in which discrepant 
evaluations of the same program are presented. Since judgments 
made by contending parties can affect the interests and prestige 
of combatants, the process of evaluation could itself become embed- 
ded in controversy. Should this occur, the public and its repre- 
sentatives, confused by competing claims, will call for clarifica- 
tion* for disinterested and objective assessments upon which they 
can rely. Controversy makes it essential that some evaluators be 
regarded as men of probity and objectivity. The educational sys- 
tem must find a way to solve the problem of integrity- trust in the 
performance of the evaluative function. 
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Processes of Innovation and Evaluation 

Two facets of the current emphasis on change deserve attention 
in our consideration of emerging issues in evaluation. The first 
is the fact of the ferment itself- -of the interest in finding and 
carrying out new ways of instructing children. The second has to 
do with the scope of changes now underway, with the emergence of 
large-scale, structural changes. Both aspects of the innovative 
thrust have important consequences for the conduct of evaluation. 

In the decades immediately preceding 1950, public education 
was characterized by a relatively slow rate of change. Despite 
the ideological concerns of the twenties and thirties, the princi- 
pal chronicler of that period does not point to consequences which 
followed in the actual conduct of school affairs. Callahan 

argues, in fact, that school practice was heavily influenced by a 

18 

simplistic conception of business efficiency. Although one can 

find changes in curricula, textbooks, teacher training, and the like 

during this period, it is difficult to identify significant struc- 

19 

tural changes. The energies of American school men and women 
were absorbed in constructing a vast system of public education 
along previously conceived lines; the years between World War I 
and 1950 were years of extending rather than reorganizing a social 
form. 

Although it is difficult to be certain (we lack empirical 
studies on the conduct of evaluation during this period) , eval- 
uation appears to have been part of a system of decision-making 
adapted to slow rather than rapid change. The thirties and 
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forties saw the emergence of "democratic administration," of a pre- 
occupation with winning the support of diverse groups for school 
activities. The superintendent, by this doctrine, should xnvolve 
a wide variety of publics in school affairs and "harmonize" them 
into a trouble-free consensus The curriculum committee is 
symbolic of this ideology. The composition of that body might 
include citizens at large, teachers, parents, professors, repre- 
sentatives of special interest groups, etc. Curriculum was not 
an area for the application of esoteric knowledge and research 
skills. One might, of course, consult an expert in evaluation or 
curriculum, but the basic mechanism in resolving issues was the 
vote rather than recourse to "professional opinion. 

Whatever the strengths and weaknesses of the participatory 
model may have been, it cannot be effectively argued that it 
fostered rapid change. Its very composition and organization con- 
forms to those bodies which Blau describes as least likely to act 
with dispatch . 21 The emphasis on consensus did nothing to advance 
techniques of evaluation. One gains the impression, in fact, that 
considerably more effort was expended in verbalizing objectives 
than in operationalizing them. Robert Wood was lead to comment, 

rather in exasperation, that "public education is a continuing con- 

"22 

stitutional convention. 

Change begets more change. For as social systems shift to 
new and different ways, solutions to problems produce new problems 
demanding new solutions . Inasmuch as education is engaged in 
serious change, pressures for more expeditious evaluation will 
mount. School officials, pressed for more rapid action, will 
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skirt cumbersome participatory methods and favor rapidly obtain- 
able expert advice. Other factors make it likely that evalua- 
tion will become defined as an area for expert treatment. The 
diffusion of the research and development orientation, coupled with 
increasing public awareness of statistics and behavioral science 
as fields in their own right, will help to define evaluation as 
"specialized" work. I consider it extremely probable that the 
expert, highly trained evaluator will come into his own. 

Some instructional changes taking place today alter student 
experience in much greater ways than in the past. Most evaluative 
work has concentrated on differences between one more-or-less 
similar component and another; one measured, for example, the 
efficacy of a given French curriculum over another. Large-scale 
changes, however, disturb aspects of student experience and social- 
ization which, previously constant, could be reasonably ignored. 
Since we did not propose to alter then, they did not matter. But 
large-scale change makes previously latent functions relevant to 
evaluative actions, for to change them without considering the 
effects on students may be to alter socialization in unintended 
ways. We can illustrate this process with concrete examples. 

The serious introduction of team teaching is manifestly a 

major change in collegial and student- teacher relationships. Yet 

our grasp of the meaning of the change for student socialization 

is limited by our ignorance of the latent functions served by the 

self-contained classroom pattern. Is Parsons right, for example, 

in implying that it requires a relationship to one nurturant 

teacher to move the student from the ascriptive world of the family 

23 

to the achievement world of the higher grades and work? 
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Could evaluators make initially positive reports on teaming in 
the early grades and miss effects which become observable, let us 
say, only during adolescence? What are the latent effects in 
grading children by age and having them move, almost regardless of 
ability, at the same rate as their age cohorts? Will self-paced 
study and nongraded arrangements, in increasing the performance 
gap between those of the same age, augment or diminish net self- 
esteem among school children? It would not be hard to produce a 
long list of such questions - -questions which point to our lack of 
knowledge about the functions of existing arrangements. How 
effectively can we assess serious change in light of our weak 
grasp on present learning structures? 

Social forms used in instruction can "contain" some values 
and exclude others apart from the explicit content communicated 
within the form. A given learning structure, I submit, may 
"instruct" persons in values considered important by the society 
yet not be explicitly planned nor consciously evaluated. Note, 
for example, how different professions use different forms in 

their professional schools without explicit theoretical justifica- 

24 

tion. Graduate departments in arts and science "automatically" 
rely on seminars and laboratories; military academies cling to 
recitation long after other instiutions have forsaken it; schools 
of architecture organize instruction around student projects, and 
medical schools elevate the importance of the clinic and operating 
room. Are such choices merely "technical" or "accidental"? Could 
it not be that the selected forms inculate, by the very rules which 
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exist within them, implicit conceptions of occupationally appro- 
priate beliefs on such questions as the relationship between know- 
ledge and action or action action rank? What underlying assumptions 
about structure and values lead graduate students, asked to design 
single-purpose schools, repeatedly to link loyalty induction to 

strict hierarchical organization, or creativity as a goal to struc- 

2 5 

tural looseness and equality? Breer and Locke have shown that 

temporary, experimental involvement in divergent task configura- 

2 6 

tions tends to change attitudes in divergent directions. Is it 
not likely that protracted engagement in particular learning struc- 
tures has considerably greater effect on student attitudes and 
values? 

The state of knowledge on interpersonal structures and social- 
ization outcomes forces us to raise questions rather than cite 
propositions. Yet the probability that structures influence stu- 
dents in as yet unknown ways is , to my view, great enough to have 
significance for evaluators . To the extent that alternative social 
forms actualize different values, evaluating "pedagogical means” 
turns out to be, in fact, the evaluation of "educational ends." 
Evaluators claiming to assess the effects of large-scale changes 
should examine functions in depth and decipher effects on latent 
as well as manifest levels. To ignore such value implications, 
perhaps by using such a single dimension as cognitive learning, 
could result in missing unintended and perhaps undesired effects. 
Large-scale changes make it inappropriate for evaluators to adopt 
a narrowly technical conception of their role, for such changes 
add moral complexities to the work of the evaluator. 



Broader Goals and Evaluative Expertise 



Schooling is more and more a matter of oroad societal concern; 
today, the specification of educational objectives includes refer- 
ences to wider social, political, and economic problems. The 
schoolhouse is no longer an isolated establishment holding interest 
only for its students, teachers, and parents; the issues which arise 
there arouse excitement in many sectors of our society. 

Examples of the newly perceived closeness between school and 
society at large are easy to find. The writer recalls that his 
undergraduate professors of economics depicted education as a 
luxury, as an activity using up scarce goods to economically ques- 
tionable ends. Today, economists pay close attention to the role 
of education in developing societies and urge heavy investment in 
it; Schultz and others argue that education contributes directly 
to human capital formation . 27 Time-worn phrases about "equality 
of opportunity" take on pungency when the federal government com- 
missions James Coleman to measure departures from that ideal in 
the conduct of public education . 28 The report that resulted 
affects our view of educational goals and processes; we are now 
more likely to concentrate on the output of self-confidence and 
the relative contribution of institutions (e.g., the family) out- 
side the formal educational apparatus. Education is involved 
deeply in other questions of our time, from structural unemployment 
to crime prevention, from producing more scientists to early iden- 
tification of emotionally disturbed children. 

It is not difficult for educators in convention to write 
statements outlining education's manifold responsibilities. It is 
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quite another matter, however, to calibrate specific instructional 
choices with particular social or economic or political goals . 

Past practice has been based on the general idea that mastery of 
conventional knowledge and/or training in a particular trade would 
result in students prepared for adult life. What happens when the 
boundaries of conventional knowledge explode? Or what decisions 
must be made when traditional occupational lines melt under the 
impact of automation? Such events make the design of study pro- 
grams extremely problematic and complicate the evaluative criteria 
to be used. The educator must become expert in gauging events 
outside of school affairs, in predicting what knowledge will prove 
basic, what core skills will have generality in the labor market, 
what educative experiences will prove to be of persisting value. 

The verbal broadening of educational objectives will make no 
discernible impact until specific instructional practices are 
aligned with specific social goals. Should demands for such 
refined’ interconnections develop, acts of evaluation will take on 
new dimensions of substantive expertise. Evaluators familiar only 
with procedures organized around in-school events will find them- 
selves puzzled in translating tests or whatever into meaningful 
indices of relevance to those demands. The question is, what sub- 
stantive knowledge will prove vital in such assessments? Should 
that knowledge prove to be various and broad, educational evalua- 
tion, as a field of expert study and practice, may itself break 
down into a series of subspecialties organized around substantive 
fields and particular societal problems. 
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Some Notes and Questions 

It is clear that the writer believes evaluative functions 
will become more critical in the years ahead; rational decision- 
making will hinge largely on whether they can be performed in an 
effective way. I wish to conclude this paper somewhat unsystem- 
atically by making additional comments on emerging issues and by 
raising a few questions which deserve the close attention of edu- 
cators. The aim is not to design a general evaluative scheme but 
to stimulate thought and discussion in the hope that those respon- 
sible for governing schools will begin work on needed solutions. 

1. We noted that there will be more options available to 
school personnel. There are forces at work which will enhance the 
role of evaluators and move evaluation toward greater expertise 
and specialization. Yet we must not overlook the great likelihood 
that all persons working in schools will be affected by the pres- 
ence of more options. Administrators, teachers, and specialists 
will perceive more personal possibilities in their respective 
roles; the public at large, long exposed to claims of professional- 
ism, will expect educators to be ready with informed judgments on 
alternatives. Scarcities in highly trained personnel make it 
unlikely that there will be enough specialized evaluators around 
to relieve other educators of all such pressures. Effective 
evaluation, furthermore, will proceed only as those who are not 
specialized come to understand enough about assessment problems 
and techniques to initiate useful questions and make sensible use 
of findings . 
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We may see a collision between the proliferation of options 

and the subculture of those working in public schools. There is, 

to my knowledge, no tradition of tough-minded empirical evaluation 

among American teachers and administrators. Their subculture seems 

29 

to stress the merits of intuitive judgment based upon experience. 

Yet experience is of little use in predicting the potential costs 
and benefits of novel alternatives. How will school people react 
to problems they cannot resolve through experience? One possibil- 
ity is firm and unyielding attachment to the status quo. Another, 
likely to occur where pressures for change are powerful, is the 
arbitrary adoption of what appear to be politic programs of action. 
The writer is willing to wager that such fadism will increase in 

the years ahead. 

Those who have a special concern with evaluation, then, face 
allocative dilemmas in making the best use of scarce teaching 
resources in their field. Granted that some upgrading of teacher 
and administrator knowledge of evaluative basics is needed, what 
weight should be assigned to that need in comparison to the pro- 
duction of able specialists? Given the massiveness of the educa- 
tional establishment and the extreme improbability of reaching two 
million teachers and administrators, which groups have the greatest 
potential for furthering effective evaluation? Presuming that 
resources will never be sufficient to find and train "enough” 
specialized evaluators, how can specialists be deployed to attain 
maximum effectiveness? In view of the long lead-time required to 
create new resources of high skill, early attention to such ques- 




tions seems indicated. 
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2. The quantity of evaluative work is likely to make it a 
routine rather than occasional activity of local school districts. 

It is 'ilso highly probable that evaluators will become key members 
of the administrative group which concerns itself with policy rec- 
ommendations. Evaluators will need considerable influence if they 
are to perform well, for they will need control over how innova- 
tions are instituted and conducted in order to generate reliable 
data. Thus, evaluators will take part in setting up record-keeping 
systems, experimental controls, etc., in order to ensure relevant 
and dependable feedback on programs and their effects. 

Constructing evaluative systems is no novelty to American 
businessmen and government officials. In business, we find elab- 
orate and precise accounting systems, production records, sales 
statistics, merchandising data, etc., integrated into overall 
statements which are highly useful in executive decision-making. 

The "art" of business management is more and more the "art" of 
interpreting quantitative data and making inferences about their 

meaning for corporate action. Federal agencies frequently pos- 

31 

sess complex machinery for evaluation and control of operations . 

The diffusion of rational modes of decision-making in public schools 
will also require the development of feedback systems useful to 
decision-making . 

Moving to the rational model is not without complication, 
however, and this is especially true in education. Business and 
government systems occur where there is little dispute over the 
propriety of hierarchical authority and organization- -both tend 
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to emphasize centralized decision-making. The introduction of 
effective evaluative controls in education could, in fact, central- 
ize decision-making without that being anyone's intention: Rourke 

believes that this is currently taking place in universities as a 
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consequence of the administrative use of computers. Ironically, 
demands for careful evaluation are arising in education at the 
point where monolithic and bureaucratic forms of administration 
are coming under attack from teacher associations, professors, and 
others. 

It would ill-suit evaluators with their passion for objective 
assessment to prejudge an issue as complex as the relative merits 
of centralized and decentralized decision-making in schools. They 
had best step gently in designing and implementing systems of data 
gathering and program control. This problem raises particularly 
vexing and subtle questions about designing evaluative systems for 
studying school programs; it looks as if special ingenuity in design 
will have to be accompanied by special understanding of the dynam- 
ics of organization and decision-making. 

3. Unchecked controversy over the conduct of educational 
evaluation could result in the loss of public confidence. Ways 
should be found to limit conflicts over the evaluative process 
itself. The issue of integrity-trus t may require considerable 
attention in the years ahead. 

The problem arises from the principle that persons and organ- 
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izaticns cannot be trusted to act as judges in their own case. 

This rule controls financial accountability in our society. No 
matter how intricate the system of internal audits or how secure 
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the reputation of officials, corporate bodies employ outside ac- 
countants to review and report on their financial status. By anal- 
ogy, we cannot expect protagonists in policy disputes or members 
of the general public to accept a school system's self -appraisals 
without question. 

Are existing organizational resources adequate to solve the 
integrity issue, or are new social forms needed? It may be, for 
example, that critical evaluations will occur infrequently and 
will be sufficiently independent of university interests so that 
professors can serve where outside, expert judgments are needed. 

On the other hand, the volume of work and the consequent necessity 
for regular "audits" may strain readily available resources in 
universities and research centers. 

What of models from other fields? Medicine, for example, has 
impressive controls based largely upon the work of pathologists in 
reviewing surgical tissue and diagnoses through post mortem exam- 
inations. But medicine is organized on the basis of sharp autonomy 
from public inspection linked to a high degree of internal, colle- 
gial control, a debatable model for schools which are part of local 
government. The role of the certified public accountant is more 
suggestive. A f ee -for-service professional, he reports in standard 
ways understandable to those who choose to learn the elements of 
accounting rhetoric. Assuming that enough work would be available 
to give them autonomy from any single client, fee-for-service 
evaluators could be employed by school boards or, in some cases, 
dissident groups, to render a public and disinterested accounting. 



22 



Such an arrangement might forestall pointless controversies where 
arguments center on the facts of the case rather than issues of 

policy. ^ 

4. Issues of moral complexity stemming from ends -mean, ambig- 
uities are difficult to resolve by examining models outside educa- 
tion. It seems that other fields can use simple dichotomies 
(profit-loss; sick-healthy) which would be gross oversimplifications 

in education. 

Could evaluative reporting, however, pay closer attention to 
this question of moral complexity by reporting empirical results 
in several ways? What I am wondering is whether alternative value 
schemes could be represented by statistical weighting schemes. 

Thus a single report might review the data gathered from several 
perspectives and in terms of several generally recognizable educa- 
tional positions. The reader would be free to introduce his own 
dichotomies if he chose; the evaluator would, on the other hand, 
avoid sacrificing complexity for M a clear answer.” 

I can see several problems in this approach. Considerable 
work would be needed to find and express moral positions which are 
meaningful to the key publics involved in public education. Open 
identification would undoubtedly stir up debate which is currently 
minimized by fuzzy statements of both goals and outcomes. But 
might the long-term gains in the quality of public discourse justify 
short-term conflicts? There is risk in the present course of 
overlooking value conflicts; evaluation may eventually suffer ”whip- 
lash” from publics who realize later that they do really want that 
particular set of values. 




An adequate system of evaluation, whatever its formats for 
reporting, will have to cope with shifts in the latent functions 
of instructional forms. But does current knowledge and research 
permit us to undertake such analysis with confidence? How much 
does social psychology tell us about relationships between social- 
ization and educational structures? If it tells us too little, 
what basic research is needed, and what responsibility do evalua- 
tors have to further such research? 

5. Broadened educational goals raise the question of evalu- 
ator expertise. To what extent should evaluators working outside 
the traditional domains of education (the economy, crime preven- 
tion, race relations) possess substantive knowledge which is 
especially relevant to the problem area? 

This is delicate territory for those who, like myself, lack 
expertise in evaluative methodology. How generally applicable are 
models used in the field? Has the historic link to educational 
psychology institutionalized data-gathering techniques and analytic 
habits that are better suited to in-school than out-school consid- 
erations? Can evaluators absorb specialized knowledge about new 
sectors and problems rapidly enough to practice on a variety of 
fronts ? 

Those who have intimate knowledge of the field are better 
equipped than I to answer these questions. The possibility of 
specialization within evaluation is, however, an issue which should 
receive very careful thought. Sho 1 such specialization prove 
desirable, it would have important implications for the training 
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o£ evaluative specialists and would point toward greater exposure 
to a wide variety of university-based disciplines. 

Speculative analyses are high-risk undertakings; there is 
no assurance that the method, no matter how prudent, discerns 
the truly vital issues . But I can conclude with one certainty. 

To crack the cake of educational custom is to release forces 
which, by comparison, make the occupants of Pandora's box appear 
to be docile, innocent, and amusing creatures. 
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FOOTNOTES 



A national survey of teachers conducted by the NEA disclosed 
that teachers come close to a representative sample of Americans 
in general in terms of the social class status of their parents. 

Many observers believe that this represents a broadening of the 
teacher base. Undergraduate schooling today is considerably 
more diverse than it was several decades ago; today there are 
relatively few beginning teachers who have been trained in ’’single 
purpose” institutions. See National Education Association, Research 
Division, ’’The American Public School Teachers, 1960-61 ,” Research 
Monograph 1963-M2. 

2 

Bagehot, Walter. Physics and Politics . New York: Alfred A. 

Knopf, 1948. 

3 

’’Evaluation” and ’’assessment” are given broad definition 
in this paper, for evaluative acts are interwined with most 
educational decisions. The writer assumes, however, that technical 
evaluation involves some measurement of effects of programs, etc., 
on students. I have omitted references to the evaluation of 
personnel since I see it as a somewhat different kind of adminis- 
trative function. 

^Callahan, Raymond E. Education and the Cult of Efficiency . 
Chicago: University of Chicago Press, 1962. 

^Compare, for example, the preoccupations of the National 
Science Foundation with those of the Office of Economic 
Opportunity. 

6 

veblen, Thorstein. The Higher Learning in America . 

New York: Sagamore Press, 1957. 

7 

One might, for example, review the activities of the 
Education Commission of the States from this perspective. 



8 

Liberman, Myron. The Future of Public Education . 

Chicago: University of Chicago Press, 1960. 

9 

Leggatt, Timothy W. The Use of Non-professionals in 
Public Education, Unpublished doctoral dissertation, University 
of Chicago, 1966. 
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10 Shapling, J. , § Olds, H. (Eds.), Team Teaching . 

New York: HarpeT and Row, 1964. 

■^Several sociologists of note have made this point. Emile 
Durkheim was among the most prominent, as in his The Division of 
Labor , Glencoe, 111.: Free Press, 1947. 

1 9 

X4 0ne of the earliest statements of this is found in 
Hughes, E. C., "Personality and the Division of Labor," in 
E. C. Hughes (Ed.), Men and Their Work . Glencoe, 111.: Free 

Press, 1958. 

13 Gardner, B., § Moore, D. Human Relations in Industry . 
Homewood, 111.: Richard D. Irwin, Inc., 1955. 

■^The author recently heard a report on this topic by James 
Guthrie presented at the Annual Social Science Institute of the 
University of California, Berkeley. 

■^There are indications that principals may become a special 
interest group. In Michigan, for example, they find themselves 
caught in cross -pressures of bargaining and are considering the 
possibility of forming their own professional association. 

■^The New York City teachers ' union was undoubtedly hampered 
by the somewhat negative report on the M. E. S. program submitted 
by the Center on Urban Education. One presumes that next time, 
they will present evaluative studies of their own! 

■^Cremin, L. The Transformation of the School . New York: 

Knopf, 1961. 

1 8 

Callahan, op. cit . 

■^A possible exception is the introduction of the junior 
high school. 

^Practically any textbook in educational administration 
published during the period will serve as an example. See, for 
example, H. Hunt § P. Pierce. The Practice of School Administration . 
Boston: Houghton-Mif flin , 1958, for a recent instance. 

^Blau, Peter. Bureaucracy in Modern Society . New York 
Random House, 1956. 

2^1 heard Robert Wood make this statement in a public address 
in Cambridge, Mass., about 1959. 
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Parsons, Talcott. "The Classroom as a Social System,” 
Harvard Educational Review , Fall, 1959. 

24 

These observations are based on a pilot study conducted by 
the author of some twenty fields in which vocational training 
occurs in universities. 
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I have asked students, subdivided into groups of four or 
five members, to design the curriculum and structure of a school 
system dedicated to one major purpose, e.g., induction of piety, 
cognitive master, creativity in the arts, etc. This has been 
done four times, and on each occasion students moved toward 
similar structures for the same overall purposes. 
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°Breer, Paul, § Locke, Edwin. Task Experience as a Source 
of Attitudes . Homewood, 111.: Dorsey, 1965. 

^Schultz, T. W. The Economic Value of Education . 

New York: Columbia University Press, 1963. 
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This impression is based on interviews conducted as part 
of the writer’s research on teaching as an occupation. 
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The Harvard Business School, long associated with case 
instruction, recently augmented the amount of quantitative material 
to be taught their master's students. There was also a special 
program instituted to train faculty members in the newer quantita- 
tive techniques. 
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Kaufman, Herbert. The Forest Ranger: A Study in Adminis - 
trative Behavior . Baltimore: The Johns Hopkins Press, 1960. 
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